Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add BigInteger type #10750

Open
wants to merge 147 commits into
base: development
Choose a base branch
from

Conversation

flashultra
Copy link
Contributor

This pull request add BigInteger type in Haxe . The implementation is created by Chuck Batson and I have his permission to relicense his code under the Haxe license ( thank you @cbatson ! ).

Some things about the current implementation. I compare three BigInteger implementations - on Chuck ( https://github.com/cbatson/hxBitcoin/tree/master/com/fundoware/engine/bigint) , in haxe-crypto lib ( https://github.com/Jens-G/haxe-crypto/blob/master/src/com/hurlant/math/BigInteger.hx ) and littleBigInt ( https://github.com/maitag/littleBigInt/blob/master/src/BigInt.hx ).

The Chuck implementation give about 5x better performance and also is better than native Java implementation ( 3x times better !).

As a additional info - many languages have a BigInteger , such as Java, C#, Javascript

@skial skial mentioned this pull request Jul 7, 2022
1 task
@Aurel300
Copy link
Member

Aurel300 commented Jul 7, 2022

Thanks, this would be useful to have in the standard library indeed. There are a couple of high-level comments I have (I haven't yet looked through all of the implementation):

  • For targets that support big integers natively (which you mention at the end of your description), can we use the native implementation? If it is in fact slower, maybe we can have a define to switch between the two versions, similar to the JSON define?
  • There should be tests, for all the API methods.
  • We might need other types to be aware of BigInt. What about the seraliser? What about additional methods to read/write in Input/Output?
  • Could @cbatson himself comment on whether the permission was indeed given to relicense the code?

@flashultra
Copy link
Contributor Author

Thank you for reviewing this pull request.
About the comments:

  • Well, this is possible, but different implementation can expose different methods, so some methods can exist only in Java, but not in Javacript , c# or to have a different names ( example isProbablePrime, modPow, ModPo ..).
    So that change will require to match any specific implementation for each language or to expose only main methods which exist in all implementations. Not sure how feasible is that solution .As I mentioned for some targets the current implementation give a better results.
    However , in the current implementation are missing some important methods such as pow , modPow and etc. , so I can try to add it.

  • Yes, I should add test. Maybe after adding the missing methods .

  • Yes, I had not thought for that, but that is valid point. At the moment is possible to convert BigInt to haxe.io.Bytes , String , Vector , which give some possibilities for serialisation .

  • His permission is public and available here : Ask for integration cbatson/hxBitcoin#4 (comment)
    I'm sure he can comment too when have a good internet connectivity

@flashultra
Copy link
Contributor Author

I think this pull request can be reviewed and merged as I added the missing methods ( abs, pow, modPow, isProbablePrime, getLowestSetBit and bitLength) and all the tests for them.
Regarding the other questions :
1) BigInt native implementation
Currently, only three targets supports BigInt type natively - Java, C# and Javascript . For the first two it contains many methods ( pow, modpow, prime ...) , but for Javascript it's just an empty class that will required other methods to be implemented using javascript's native BigInt type .I'm not sure how feasible it is since the current implementation stores data in a Vector. For C# and Java - they do not support Muttable BigInt type (this PR adds immutable and mutable types - BigInt and MutableBigInt) . It might be possible to add only muttable as native type, but maybe in the future if someone is interested in that.
2) Serializer
There are several solutions for that - for example, like adding BigInt as a separate type to the Serializer class, perhaps similar to StringMap, IntMap, ObjectMap, which would require a change to the Serializer class, or simply serializing the BigInt as a class or bytes (there is a method to convert to bytes) .
This may be a separate PR in the future, and as I mentioned, it is currently possible to serialize a BigInt to Bytes.
3) Input/ Output
Internally BigInt keep the data in Vector. This could be ease converted to Bytes.
So I think it's possible someone to get the data and use it in Input/Output methods .
Another solution could be adding new methods writeBigInt / readBigInt ( with keeping the size of the Bigint ) , which will require change in Input /Output class in separate PR.

@cbatson
Copy link
Contributor

cbatson commented Jul 18, 2022

Yes, this does indeed have my blessing. Hope it's a useful addition. 👍

Thank you @flashultra for doing this work!

@flashultra
Copy link
Contributor Author

Php tests fails on another test. I think the bigint tests should pass the Php target.

@flashultra
Copy link
Contributor Author

All tests are green except PHP ( and mac-cpp which failed with other reason) .
For php test I tried to run on my server , but I've got the following error which is not related with BigInt.
I think it's safe to be merge this pull request.

Stack trace:
#0 /var/www/lib/Array_hx.php(11): php\Boot::php\{closure}()
#1 /var/www/index.php(9): include_once('...')
#2 /var/www/lib/unit/Test.php(629): {closure}()
#3 /var/www/lib/unit/Test.php(644): unit\Test::__hx__init()
#4 /var/www/index.php(9): include_once('...')
#5 /var/www/lib/unit/TestBigInt.php(23): {closure}()
#6 /var/www/index.php(9): include_once('...')
#7 /var/www/lib/BigIntTest.php(16): {closure}()
#8 /var/www/index.php(14): BigIntTest::main()
#9 {main} in /var/www//lib/Array_hx.php on line 11
PHP Fatal error:  During inheritance of JsonSerializable: Uncaught ErrorException: Return type of Array_hx::jsonSerialize() should either be compatible with JsonSerializable::jsonSerialize(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /var/www/lib/Array_hx.php:267

@flashultra
Copy link
Contributor Author

The above error is when I test with PHP 8.1.
I ran a new test with PHP 7.3 and all tests passed.
This PR is ready to be merged.

@flashultra
Copy link
Contributor Author

@Simn About this pull request is there any chance to be merged with Haxe 5.0 ( as this is a major change ) ?
Some consideration:

  1. Lua finally works;
  2. Php and Python have a random fails which should be fixed with [JS] Fix for Issue11475 #11868 accoding my investigation
  3. Neko fails with a "module.c(560) : Stack check failed for function scope" . I do not know why, but is Neko still be relevant in Haxe 5 ?
    All other targets looks ok.

@tobil4sk
Copy link
Member

Neko fails with a "module.c(560) : Stack check failed for function scope" . I do not know why, but is Neko still be relevant in Haxe 5 ?

This means that there is a function somewhere with too many variables in the same function scope for neko to be able to handle it. This is an updated error message for this issue which you reported: HaxeFoundation/neko#283.

@flashultra
Copy link
Contributor Author

Thank you.
It's probably related to Int64 again. Dealing with this will require some deep investigation and possibly adding separate scopes using {} blocks.
But is Neko still relevant in Haxe 5 and should it be supported with new functionality (what would BigInt be) ?

@kLabz
Copy link
Contributor

kLabz commented Dec 16, 2024

Yes Neko is still relevant for Haxe 5.

It should be possible to include BigInteger in Haxe 5; maybe we should discuss it during our next haxe-evolution meeting

@flashultra
Copy link
Contributor Author

flashultra commented Dec 16, 2024

Does this mean that BigInt should be supported in Neko ?
As I mentioned, this will require debugging all BigInt methods and adding a lot of "#if neko" defines for every function where, probably, there is an arithmetic operation with three or more Int64 variables.

@flashultra
Copy link
Contributor Author

Almost all is green ,except for the CPP target ( timeout ).
Maybe the tests are too agresive for some targets ?
Most of the targets keep almost the same time to finish the test as before BigInt test, except for:

  1. Php - before 1min 8sec , now 4min 55sec
  2. Python - before 1min 30sec, now 7min 25sec
  3. Lua - before 5 min 48sec , now 44min 12sec
  4. Cpp - timeout ( Is cpp the problem or cppia ?)

@flashultra
Copy link
Contributor Author

I think everything is fine now. Cppia failed with an error "terminate called after throwing an instance of 'Dynamic'" which I don't think is related to this pull request.

@flashultra
Copy link
Contributor Author

All is green except Cppia with "-jit" option on Linux and Mac only.
All errors in cppia jit leads to simple vector assigment such as m_data.set(pos, m_data.get(pos) | (c << bit)); which is very hard to reproduce in a simple test program.
It always happend in BigInt test file , but it's not possible to extract the reporoducable code.
For anyone intresting to repat the erorr just call testBigIntTools(); and in MutableBigInt_.hx function _setFromHex( the row m_data.set(pos, m_data.get(pos) | (c << bit)); will return the error terminate called after throwing an instance of 'Dynamic' ( on Linux and Mac only)
I won't do any more changes on this request and could not fix cppia jit erorr.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants